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ABSTRACT 


This study analyzes the applicability of software cost 
estimating models to a tactical information processing system. 
In particular, the Boehm and Putnam models are used to obtain 
predictions which are compared to contractor estimates for 
the TRI-TAC AN/TTC-42 Unit Level Circuit Switch. Factors 
affecting model performance are also examined. Conclusions 
address the requirement for more accurate estimation of soft- 
ware project scope and clarification of model input parameter 


definitions. 
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I. INTRODUCTION 


Eiince the Intheamettom Of the stored program computer, 

a reversal in the proportion of development cost attributed 
to hardware and to software has occurred (Boehm: pp. 17 §& 
ieee L981). Unfortunately this increase in the contribution 
of software to overall development cost has not been accompa- 
nied by a similar increase in the accuracy of software cost 
and schedule estimating procedures (Putnam: pp. 13 § 14, 
metelhibodeau: pp. 5-1 | 5-2, 1981; ITT: Sec 400.22, 
1981). Two of the models which have been proposed to remedy 
Mies Situation will be the subject of this study. The 
Constructive Cost Model (COCOMO) (Boehm, 1981) and the Putnam 
Model (Putnam, 1980) will be applied to an ongoing automated 
System development program in an effort to assess their 
applicability in the study environment. 

The vehicle for the study will be the AN/TTC-42 tactical 
telephone switching central, which together with a smaller 
Switchboard, the SB-3865, comprises the Unit Level Circuit 
Switch (ULCS) family. These two equipments are designed to 
provide "reliable, ... and highly mobile telephone switching 
equipments for ‘unit level’ organizations such as division 
and brigade..." (Huebner-Wright, 1980). In many ways they 
resemble the private automatic branch exchanges (PABX) that 


have become common in fixed plant telephone installations 





during the last decade, but they possess additional capa- 
bilities needed on the modern battlefield, for example, 

high mobility and cryptographic protection. Both switch- 
BOards, or circuit switches as they are officially known, 

are being developed by the U.S. Marine Corps as part of a 
Department of Defense mandated move away from the analog, 
non-secure, and often incompatible telephone systems current- 
ly in use by U.S. military forces to a digital cryptographi- 
cally secure and interoperable environment. If this change 
Seemrred too rapidly a large portion of the existing tactical 
telephone plant would have to be discarded well before the 
end of its useful service life. To avoid this a multi-service 
effort, the Joint Tactical Communications (TRI-TAC) Program, 
is developing an integrated system of hybrid analog and 
digital tactical circuit switches, including the AN/TTC-42, 
which will be able to operate in today's analog environment, 
in the mixed analog and digital transition period and finally 
in the all digital scenario. 

Placing the burden of compatibility on the TRI-TAC 
systems may appear to be a simple solution to the problem of 
integrating a wide variety of equipment into a common system, 
but implementing that simple concept requires complicated 
Solutions. When this requirement is tied in with others such 
as faster and more reliable service, and cryptographic pro- 


tection, a purely hardware-oriented solution is infeasible. 
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In the AN/TTC-42, the hardware is complex, but the complexity 
has been held to tenable levels by transferring much of it 

to software. Two Intel 8080A microprocessors, one each for 
the switching (SCPU) and monitor (MCPU) control fucntions, 
are used to operate the switch as a real-time, event-driven, 
table-controlled system. The programs required to accomplish 
this are quite complex, and would be much larger than they 
are but for the use of even more complex control tables. 

When an event occurs that requires processing by one of the 
microprocessors, an interrupt is generated to the unit con- 
cerned. The unit identifies the element of its environment 
that initiated the interrupt and the point in the unit's 
program where processing is required to begin by means of 

the control tables. This technique saves processing time and 
memory space, but does so at the cost of utilizing some very 
sophisticated and difficult-to-develop software. 

The Naval Electronic Systems Command (NAVELEX) acts as 
the ULCS program manager for the Marine Corps. NAVELEX in 
turn is aided in the performance of its ULCS responsibilities 
by a support contractor, the Federal Systems Group of Calculon 
Corporation. Actual development of the circuit switches is 
being conducted by the International Telephone and Telegraph 
Corporation's Defense Communications Division (ITT) under a 
contract awarded in August 1977. The contract price was 


$19.5M, and ITT agreed to contribute another $5.4M in cost 





Pino liveso cOstestaring Orter Made ITT's excellent tech- 
nical proposal even more appealing, and was reputedly made 
by the company in a bid to expand its technological base. 
Completion was scheduled for thirty-six months from the 
Sentract award date. 

ITT prepared a highly sophisticated environment for the 
development of the ULCS software. ITT software design 
motlosophy (1TT, 1981) places heavy emphasis on the complete 
identification of software performance requirements before 
the software design phase commences. Top-down design is 
stressed and modular integrity 1S maintained to the maximum 
degree permitted by performance requirements. Structured 
programming is used to implement the top-down design and 
Mmmoegram design language (PDL) is employed to enhance modu- 
larity and to ease the transition from the program design to 
code. The code itself is written by chief programmer teams 
in XAS-8 assembly language, a Pascal derivative that lends 
itself to structured programming techniques. 

Extensive software tools were developed and employed to 
implement this design and programming philosophy. A UNIX 
interactive timesharing system is used to edit code, main- 
tain program files and to generate program documentation. 
The XAS-8 source code is translated to Intel 8080A object 
code by a macro assembler, and a link editor is employed to 


eesemble object routines for testing. The SCPU, MCPU, their 





interfaces and hardware environments are emulated on a DEC 
PDP 11/70 computer. This has permitted extensive testing of 
the software before hardware completion. In addition, an 
environmental simulator is used to construct precisely 
controlled and completely reproducible test scenarios. 
Configuration control and resource (memory and processor) 
utilization are continually scrutinized, and an independent 
quality assurance agent is employed. 

MOtwitniscHndim= tte oopnistication of the development 
environment, as the ULCS project progressed it became apparent 
that cost overruns and schedule slippages were occurring. 
These resulted in an ITT proposal for rephasing the project 
(known in the program jargon as rebaselining) in April 1979, 
and another two years later in July 1981. The 1979 rebase- 
lining was considered to be the result of project growth and 
increased project scope in the ratio of two to one (Crandell: 
meet, 1981), with software among the critical path items. 
The contractor seems to have seriously underestimated both 
the size and the difficulty of the software development for 
the ULCS in its contract proposal; so much so that even 
though the design was frozen after the 1979 rebaselining, 
another rephasing was required by 1981. This time software 
was considered responsible for one-half of the overrun 
maoorem (Crandell: p. 33, 1981). The project's history 


through the 1981 rebaselining is summarized in Table 1 where 
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TABLE 1 


UECS PROGRAM HISTORY (NAVELEX: 1977-1982; CRANDELL, 1981) 


Year * Est Prgm Size # Est Dev Cost # Est Dev Sched 
(Kbytes) ($M) (Mos) 
9 7 7 91 Ave 24 
1979 ty 6.8 48 
1981 ASS 14.0 66 
# Remaining Software # Remaining Software 
Budget ($M) Schedule (Mos) 
1977 4.1 24 
1979 a0 29 
1981 4.8 19 


@ AN/TTIC-42 


# AN/TTC-42 and SB-3865 


ULCS software development cost and schedule at contract award 
and at each of the rebaselinings is presented along with the 
estimate of AN/TTC-42 program size (SCPU plus MCPU) for those 
points. In addition, the estimated remaining development 
cost and schedule at each point are also presented. Although 
the cost figures for SB-5865 software development could not 
be separated from those for the AN/TTC-42, some idea of the 
error introduced may be gleaned from the AN/TTC-42's position 


fae SOttware parent” to the smaller circuit switch. Of an 
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estimated 2408 ULCS software routines only 210, or 8.7 per- 
cent, are SB-3865 unique. The remainder were either adopted 
whole or modified from AN/TTC-42 routines. 

The primary intent of this study is to discover the 
extent to which software estimating models such as COCOMO 
and Putnam are applicable in development scenarios similar 
to that of the ULCS. The predictions of all three models 
(ITT, COCOMO, and Putnam) will be compared in the hope of 
gaining some insight into the software estimating process. 
Although the models possess features in addition to cost and 
schedule estimation, only these will be examined in detail. 
In particular, techniques for estimating program size will 
not be considered. The estimates of program size produced 
by ITT will be accepted as having formed the basis for its 
cost and schedule estimates, and are therefore considered 
valid for generating similar estimates with the COCOMO and 
Putnam models, Finally, development cost will be measured 
only in units of labor, for example, man-hours. No attempt 


1s made to monetarize these values. 


es 





If. MODEL DESCRIPTIONS 


fee, CATEGORIZATION OF MODELS 

The two models for estimating software cost and develop- 
ment time that are the subject of this study and the model 
used by the development contractor will be described in this 
chapter. The description of each model will consider its 
basic estimating methodology; origin and source data base, 
mE known: required inputs; the evaluation process, and model 
outputs. In addition, model features that are beyond the 
scope of this study will be briefly summarized for the sake 


of completeness. 


mee lHE ITT MODEL 

The ITT cost estimating model is described in (ITT: 
meema00,22.2.6, 1981). Although the cover sheet of (ITT, 
1981) identifies the publication date as August 1981, the 
opening sections are dated 1980, and the software cost esti- 
Mating section is dated October 6, 1981. This researcher 
was unable to determine the exact extent to which (ITT, 1981) 
reflects the cost estimating methodology used for the ULCS 
contract proposal and the two rebaselinings, but conversations 
with contractor software personnel persuaded this researcher 
that even if particulars have changed, the underlying philos- 
ophy and general methodology remain the same. As will be 


described below, this philosophy stresses disassembly of the 
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fPoatware Droduet into the smallest possible components. This 
procedure is followed in the belief that the estimating 

error associated with these smaller and presumably better 
understood elements is proportionately smaller than that of 
larger elements. The estimating philosophy also stresses 

the requirement for an extensive data base of carefully 
collected information from previous projects. It notes that 
estimates of software development efforts are meaningless 
unless they can be calibrated against previous performance. 
(Unfortunately the researcher was not able to discern the 
composition of the current ITT data base.) Finally, ITT 
rejects the existence of an algorithmic solution to the cost 
estimating problem, and stresses the role of individual 
judgment in software cost estimating to a much greater degree 
than do COCOMO and Putnam. Interestingly, although (ITT, 
1981) seems quite concerned with estimating the amount of 
effort required to develop software products, the model 
assumes that scheduled development time is a given. This 
Corresponds with industry practices described in (Boehn, 
1981) and (Putnam, 1980). 

The ITT procedure requires as an input only the overall 
number of lines of code to be produced by the project. The 
Bote lines O£ code“ 15 undefined in (ITT, 1981). No 
differentiation is made between executable lines of code, 
data declarations or comments. Further, there is no 


reference to the language level used for project estimating 


14 





pumposes. The procedure does identify a quantity referred 

to as "Total Software Effort" measured in lines of code and 
divisible into support, operational and hardware support 
categories. The support category encompasses the software 
required to produce deliverable programs and data bases. It 
includes such items as "compilers, linkers, editors, debuggers," 
and "test environment." The operational category consists of 
the deliverable products themselves, examples of which are 
"operating systems, communications interfaces, interrupt 
handlers, on-line diagnostics," and "application data base 
processing.'' The hardware support category supports the 
development of new equipment by assisting the efforts of 
departments other than software engineering (ITT: Sec 400. 
meercz.6, p. 4, 1981). 

Given an input in terms of lines of code separated into 
the categories described above, the ITT estimating process 
Meri. Sec 400.22.2.6, pp. 4-6, 1981) requires that each 
category of software be disassembled until no sub-item is 
either more than 5 percent of its category or 1000 lines of 
code if the total software effort is less than 20,000 lines 
of code, The next step is to compute a "Project Difficulty 
Factor" (PDF) using the following algorithm: 

1. An "Instruction Mix Weight" is assigned to each sub- 
item. This value is obtained from a table which forms part 
of the RCA Price S software estimating model. It represents 


the relative difficulty of producing software for such 


a, 





applications as operating systems and mathematical 
operations. 

2. The percent of total software effort represented 
by each sub-item is determined. 

5.) oie “Preject Difficulty Factor’ (PDF) 1s computed 


according to the formula: 


n 
PDF = £ (w-)(%.) , (1) 
acme 
where: Ww. = the Instruction Mix Weight of the ith 
sub-item, and 
35 = the percent of the total code represented 


by the ith sub-item. 


iembowing Ccalcttlation of the "Project Difficulty Factor," 
the relative complexity of the project under consideration 
memadetermined by computing a “Project Complexity Factor" 
(PCF) for each sub-item using another facet of the RCA Price 
9 model, "Complexity Adjustment" values. These values are 
provided for personnel, product familiarity and complicating 
factors. For personnel, the ratings range from "Outstanding 
crew" to "Relatively inexperienced." Product familiarity 
Similarly ranges from "Old hat, redo of previous work'' to 
"New line of business,'' and complicating factors includes 
such items as "New hardware" or "Hardware developed in 
parallel.'' When these values have been determined they are 


entered into equation (2) to obtain the complexity factor. 
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PCF = 1.0 + PV + PFV + CEV, (2) 


where: PV = Personel value, 
PFV = Product familiarity value, and 
GEV >= Commllexity tactor value. 


When the preceding steps have been completed, an estimate 
of the effort (man-hours) required to develop each sub-item 
of code is generated based on a comparison of the Project 
Difficulty and Complexity Factors with the historical data 
base. Although no specific directions are presented for 
making this comparison, it seems to be highly subjective 
as the estimator is cautioned to include notes "that describe 
the thought process for determining the estimate" in the 
project documentation. To arrive at an overall effort 
estimate, the sub-item estimates are summed with independently 
determined estimates for non-coding activities such as manage- 
ment, and the development of test plans and documentation. 

The non-coding estimates are supposed to be based on the 
amount of coding effort required, but no specific directions 
for determining the non-coding effort values are given. 

Olio 198) dees not specify the periods within the 
development process that are included by the effort esti- 
mating methodology, but the data obtained for this study 
include all effort estimated for the software portion of the 


project work breakdown structure (WBS) from the drafting of 





the Program Design Specification (PDS) to completion of the 
software integration and test phase. The Program Design 
Ppectiilcation is a key project document which is described 
mee tollows in (ITT, 1981): 
The Program Design Specification (PDS) describes the 
Structure and design of the software necessary to 
implement the operational, performance and functional 
requirements defined in the Program Performance Speci- 
fieation. the program architecture is defined in the 
PDS, and the programming approach for developing the 
software is specified. 
Software integration and test involves the assembly of the 
entire deliverable software product, and verification that 
it meets all of the specified requirements. These points 


memeesclected as Seeming to best meet the input criteria of 


the COCOMO and Putnam models. 


C. COCOMO 

iiiemconstructiure COst MOdel or COCOMO (Boehm, 1981) is 
a comprehensive software life-cycle cost and schedule esti- 
mating model. The primary emphasis of the model is on the 
development portion of the life-cycle which COCOMO defines 
as beginning "at the beginning of the product design phase.. 
and end({ing) at the end of the (software) integration and 
test phase." (Boehm: p. 59, 1981) This definition seems 
consistent with that used to gather the ITT data on which 
the study will be based. COCOMO can be employed at three 
levels of sophistication which in increasing order are 


Basic, Intermediate and Detailed. The Basic and Intermediate 
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models are utilized in this study, but insufficient project 
data was available to allow the Detailed model to be employed. 
COCOMO was derived from a study of sixty-three widely 
varied software development projects which are summarized in 
(Boehm: pp. 496 §& 497, 1981). Boehm describes the method 
by which he arrived at his estimating relationships in the 
following quotation: 
The calibration and evaluation of COCOMO has not relied 
heavily on advanced statistical techniques. After try- 
ing to apply advanced statistical techniques to software 
cost estimation, and after observing similar efforts by 
others, I have become convinced that the software field 
1s too primitive, and software cost driver interactions 
too complex, for standard statistical techniques to make 
much headway; and that more initial progress could be 
made by trying to formulate empirically the nature of 
the interactions between cost drivers, using functional 
forms which reflected the best available perspectives 
and data on software life-cycle phenomenology. (Boehm: 
p. 493, 1981) 
In contrast with the ITT model, COCOMO is an algorithmic 
model which posits definite relationships between model 
inputs and outputs, although depending on the version of 
COCOMO used, individual judgment may still have a substantial 
impact on model performance. The Basic model is almost 
meee tely determinastic, and only one input requires the 
exercise of judgment on the part of the user. The Intermediate 
and Detailed models provide increasing amounts of scope for 
the application of user judgment. 
1. Basic COCOMO 
PeasciemeeeOrl® “boeehm: ppys57-115, 1981) 1s both the 


least sophisticated and the least accurate of the three 


is) 





COCOMO models. For inputs it requires only the number of 
thousands of delivered source instructions in the software 
product and the mode in which the product is being developed. 
Delivered source instructions are defined (Boehm: pp. 58 & 
ee, 1981) as follows: 
Delivered. This term is generally meant to exclude 
nondelivered support software such as test drivers. 
However, if these are developed with the same care as 
delivered software, with thelr own reviews, test plans, 
wOcumentation, etc., then they should be counted. 
Source Instructions. This term includes all program 
instructions created by project personnel and pro- 
cessed into machine code by some combination of pre- 
processors, compilers, and assemblers. It excludes 
comment cards and unmodified utility software. It 
includes job control language, format statements, and 
data declarations. Instructions are defined as lines 
of code or card images. Thus, a line containing two 
Or more source statements counts as one instruction; 
a five-line data declaration counts as five 
instructions. 

By software development mode Boehm means a general 
description of the software product and its development 
environment. Three modes are defined -- organic, embedded 
and semi-detached. Organic projects are characterized by 
relatively small development teams having extensive experience 
with systems similar to the one under development, and opera- 
ting in a stable, familiar, in-house environment. No 
particular importance is attached to early completion of the 
project. An embedded mode project is developing a product 


that "must operate within (is embedded in) a strongly coupled 


complex of hardware, software, regulations and operational 





peocedures....'' A great deal of emphasis is placed on 

early or at least timely completion of the project as 

schedule delays may result in large cost overruns. The 
semi-detached mode is characterized either by an intermediate 
level of project characteristics or by a mixture of organic 
and embedded mode characteristics. The development mode 
determines which of three sets of cost and schedule estimating 
equations (Boehm: p. 75, 1981) is used in the Basic model. 

In the case of the AN/TTC-42 the rigid interface specifications 
between the circuit switch software and TRI-TAC software, as 
well as the equally inflexible interfaces between the circuit 
Switch hardware and software place the development effort 

in the embedded mode. 

Once the number of thousands of delivered source 
instructions (KDSI) and the eee pnieat mode have been deter- 
mined, an estimate of project software development effort 
(MM) in man-months (1 man-month = 152 hours of working time) 
can be made uSing equation (3) in the case of the embedded 


mode, 


0 


MM = 3.6 (KpS1)+°4 (3) 


In addition to providing estimates of software develop- 
ment effort and schedule, Basic COCOMO has two other features 
of note. It provides a breakdown of the development effort 


and schedule estimates by phase and it provides a means for 


oe 





estimating the amount of effort required to maintain the 
software once it has been developed. Schedule and effort 
distributions over the three phases covered by the model 
(Product Design, Programming, and Integration and Test) and 
the Plans and Requirements phase are provided for several 
project sizes. The distributions for intermediate size pro- 
jects are obtained by linear interpolation. Estimated 
Annual Maintenance Effort is determined based on the amount 
of effort required to produce the software and a parameter 
known as Annual Change Traffic, which is defined as, "The 
fraction of the software product's source instructions which 
undergo change during a (typical) year, either through addi- 
maomeor moOdification.'’ (Boehm: p. 71, 1981) 
2. Intermediate COCOMO 

Intermediate COCOMO (Boehm: opp. 114-163, 1981) pro- 
vides increased eStimating accuracy over that available from 
Basic COCOMO by using two additional inputs, Equivalent 
Delivered Source Instructions (EDSI) and an Effort Adjustment 
Factor (EAF). The primary input to the model is still 
thousands of delivered source instructions (KDSI), but pro- 
vision is made in the Intermediate model for differentiating 
between newly developed software and existing software 
adapted for use in the project. The model provides an algo- 
rithm that converts the quantity of adapted code into an 


equivalent number of new delivered source instructions (EDSI) 





based on such factors as the percent of the adapted software's 
design that required modification in order to function in the 
new environment. When adapted software is used on a project, 
the Equivalent Delivered Source Instructions are divided by 
1000 and summed with KDSI to provide a new effort estimating 
equation input -- KEDSI. The AN/TTC-42 uses no adapted 
software; so in this case KDSI equals KEDSI. 

The other new input, the Effort Adjustment Factor 
(Boehm: pp. 117-120, 1981) provides a means of modifying the 
amount of development effort estimated to be required for a 
project based on fifteen cost driver attributes which are 
listed in Table 2. Cost drivers are generally rated on a 
scale of very low to extra high although some do not have 
values at one or both extremes of the scale. Once the soft- 
ware cost driver ratings have been determined, the associated 
Software Development Effort Multipliers (SDEMs) are extracted 


meom (BOchm: p. 118, 1981). The Effort Adjustment Factor 


b 


can then be calculated with equation (5). 
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BAF SDEM, (5) 


1=1 
In developing Intermediate COCOMO development effort 
estimates the Basic COCOMO relationship, equation (3), is 


retained but with a different coefficient which was empiri- 


cally determined by Boehm to provide better results in the 





TABLE 2 
MOGOMO COST DRIVER ATTRIBUTES (BOEHM: PP. 115 §& 116, 1981) 


Product Attributes 

RELY Required Software Reliability 
DATA Data Base Size 

CPrx Product Complexity 


Computer Attributes 


TIME Execution Time Constraint 
STOR Main Storage Constraint 
Vater Virtual Machine Volatility 
TURN Computer Turnaround Time 


Personnel Attributes 


ACAP Analyst Capability 


AEXP Applications Experience 

PCAP Programmer Capability 

VEXP Virtual Machine Experience 

LEXP Programming Language Experience 


Project Attributes 
MODP Use of Modern Programming Practices 


TOOL Use of Software Tools 
SCED Required Development Schedule 


Intermediate model. The general case of the Intermediate 
model is one in which there is no adapted software and all 
of the cost driver ratings are nominal. The Software 
Development Effort Multiplier for a nominal cost driver 
rating 1s one, which by equation (5) means that the Effort 
Adjustment Factor for the general case is also one. The 


feeubting Intermediate COCOMO nominal effort, (MM) vou 
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estimating equation is equation (6). 
eco 
(MM) Nom = 2.8 (KDSI) (6) 


For cases in which adapted software is utilized or for which 
non-nominal software development cost driver ratings exist, 


adjusted software development effort, (MM) 1s computed 


DEV’ 
by equation (7). 


0 


(MM) pey = 2.8 (KEDSI)**°" (BAF) (7) 


The Intermediate COCOMO software development schedule (TDEV) 
estimating equation is equation (8). It is the Basic COCOMO 
relationship modified by the substitution of adjusted develop- 


ment effort (MM) ney for development effort (MM). 


.32 
ioEVe= 2.5 (MM) yey (8) 


In addition to estimates of software development 
effort and schedule based on overall program size, Intermediate 
COCOMO provides procedures (Boehm: pp. 146-152, 1981) for com- 
puting these same estimates from smaller units of code called 
Components. Boehm does not define the term, but directions for 


employing components in the estimating process and examples in 


cA 





(Boehm, 1981) indicate that it refers to a major software 
subdivision one step below the overall product. Examples 
provided in (Boehm: p. 155, 1981) for a communications front 
end processor include task control, communications, and status 
Memitoring. As it is unlikely that the cost driver ratings 
for all of the software components will be identical to that 
moe the overall product, the effect of the use of this two- 
level estimating procedure should be to increase model 
accuracy. 

In an extension of the Effort Adjustment Factor 
concept, Intermediate COCOMO enables the user to compute an 
adjusted estimate of Annual Maintenance Effort (Boehm: 
pp. 129-132, 1981) utilizing the cost driver attributes in 
Table 2 with the exception of Required Development Schedule. 
The Intermediate COCOMO distribution of effort by phase and 
schedule is the same as in the Basic model. 

5. Detailed COCOMO 

Detailed COCOMO contains two extensions to the 
Intermediate model, phase-sensitive effort multipliers and a 
three-level product hierarchy. In Intermediate COCOMO, 
effort multipliers are assumed to have one value applicable 
over the life of the project. The Detailed model provides 
discrete effort multiplier values for the cost drivers 
during each phase of the project. The Detailed model also 


utilizes a three-level product hierarchy of module, subsysten, 
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and system in place of the two-level hierarchy used in the 
Intermediate model. At the lowest level, the module, the 

cost drivers most likely to affect the output of individual 
programmers are applied, while the remaining cost drivers 

are used at the subsystem level. These lower level estimates 
are aggregated and used to produce project estimates of effort 
and schedule at the system level. The estimated Annual 
Maintenance Effort is identical to that of the Intermediate 
model, and the phase and schedule distribution of effort is 


the same as that in the Basic model. 


D. THE PUTNAM MODEL 

The Putnam model comprises those portions of a proprietary 
software development system, the Software Life Cycle Model 
(SLIM), that have been placed in the public domain (Putnam; 
Thibodeau: pp..A-63 - A-80, 1981). The general principles 
of the proprietary system are the same as those in the Putnam 
model, but SLIM contains additional features which improve 
estimating accuracy, particularly for projects of less than 
70,000 source statements (Putnam: pp. 319 § 320, 1980; 
Thibodeau: p. A-68, 1981). The model is based on the premise 
that the rate at which effort is expended in the solution of 
development problems follows a so-called Rayleigh-Norden dis- 
tribution (Putnam: pp. 17 § 18, 1980). Putnam's development 
of this concept while at the U.S. Army Computer Systems 


Command and also with data from the U.S. Air Force's Rome 
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Air Development Center (RADC) led to the formulation of the 
model (Putnam: pp. 32 §& 37, 1980). 

The model draws its utility from the proposition that 
Within defined limits, software development efforts (E) and 
development time (ta) can be substituted for each other. 
The relationship describing the rate at which one may be 
traded for the other is described by the Software Equation, 


equation (9). 


ty (9) 


The Software Equation requires as inputs the amount of 
effort to be expended in developing the software (E), the 
length of time allotted for software development (ty), and 
the technology constant (C,). The nominal output of the model 
1S program size in source statements (S.), although as noted 
above the more likely case uses inputs of program size, 
technology constant, and one of the remaining parameters to 
determine either required development effort given the 
development time or development time given effort. 

The example of the Software Equation given in equation 
(9) uses the effort expended during the development period 
as the input parameter. This version of the equation was 
used as development effort in one of the parameters of 


interest in this study. The more common form of the Software 
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Equation uses life cycle effort (K). The conversion from 
life cycle to development effort is accomplished using 


equation (10). 


Ee tk) C05 


This relationship is based on the premise that the maximum 
rate of expenditure of effort on a project occurs just prior 
meeene Completion of the development period (Putnam: opp. 7, 
mG 514, 1980). By analyzing the project data available to 
him, Putnam determined that at this point forty percent of 
the life cycle effort has been expended. His concept of what 
constitutes development time (t 4) does not extend much beyond 
the explanation offered above for development effort. The 
figure contained in (Boehm: p. 314, 1981) shows two phases, 
design and coding and test and validation, in the development 
period, but provides no additional information concerning 
their composition. In addition, a portion of the system 
installation effort is also contained in this period. Insuf- 
ficient information concerning the model is available to make 
a judgment concerning the degree to which the available ITT 
data meet the model criteria. 

The technology constant (Cy) is a dimensionless value 
"which 1s somehow a measure of the state-of-technology being 


Meeered to the project.” (Putnam: p. 168, 1980). It “is 
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obtained by calibrating the model using historical data that 
are representative of the project to be estimated" (Thibodeau: 
p. A-77, 1981), and "measures any throughput constraints that 
impede the progress of programmer/analysts.... Cy is 
quantized...fand) varies in a set sequence of allowable 

values (Fibonacci sequence.)"' (Putnam: p. 7, 1980) 

The Software Equation 1S not applied in a vacuum. As 
noted above the degree to which effort and development time 
may be substituted for each other is not unlimited. For 
each type of development undertaken by an organization, for 
example, “stand alones" and "'rebuilds,'' there is a relation- 
ship of the form given in equation (11) that defines the 
aimerrculty Gradient’ (VD) for that particular organization 


and type project. 


The difficulty gradient defines the minimum feasible combina- 
tions of development effort and time required to produce a 
software product of a given size. It is derived from a study 
of the software development organization's previous performance, 
In addition to the Software Equation, the Putnam model 
demonstrates an expected value technique for estimating 
project size from independent estimates of the largest 


possible, least possible and most likely project sizes. The 
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model also demonstrates the use of the Monte Carlo method for 
determining the sensitivity of the model to uncertainties 

in the model inputs. Linear programming solutions to develop- 
ment effort and schedule optimization problems are discussed, 
as are techniques for determining optimal project man-loading. 
An example of dynamic project cantrol using differential 
equations is presented, and the elements for a software cost 


estimating data base are recommended. 





IIIT. DATA AND EVALUATION PROCEDURE 


ies datea evaluated in this study are listed in Appendix A. 
They comprise three data points which were extracted from 
the 1979 and 1981 rebaselining documentation (NAVELEX, 1977- 
1982) by the Naval Electronic Systems Command's support 
memeractor, the Federal Systems Group of Calculon Corporation. 
Two of the data points (1977 and 1979) provide estimates of 
program size, development effort and schedule. The third data 
point (1981) includes estimates of program size and develop- 
ment schedule only. In addition to these data points a stand 
along estimate of program size (Crandell, 1981) for 1981 was 
obtained. The Crandell estimate was obtained from a report 
to the Director of the Joint Tactical Communications (TRI-TAC) 
Office concerning the December 1981 status of the ULCS pro- 
feem, ine report contains a recapitulation of the project's 
Nnistory, discusses the probable results of continuing the 
existing program management approach, and makes recommenda- 
tions for improving program performance. 

UwGoeprogram Size estimates are maintained by ITT in 
lines of program design language (PDL) and bytes. Program 
design language is a pseudo-higher order language which aids 
programming personnel in applying the principles of top-down 
design and structured programming. In the case of this 


project it is closely related to the formal programming 
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language (XAS-8) and supposedly provides a smoother transi- 
tion from program design documentation to the actual programs 
than is provided by such techniques as flow charting. 
Program size in lines of PDL was determined by count. 
Program size in bytes was also determined by count and is used 
to generate two additional estimates in terms of 8080A 
instructions and lines of XAS-8 assembly code. Program 
estimates in bytes are convertible to 8080A instructions at 
the rate of 1.8 bytes per instruction, and to lines of XAS-8 
at the rate of 3.1 bytes per line (NAVELEX, 1977-1982). The 
8080A conversion rate is based on an "average" 8080A instruc- 
tion size. Actual 8080A instructions occupy one, two or three 
bytes. The XAS-8 figure was obtained from a NAVELEX study 
of a sample of the source code and its corresponding object 
code. The Crandell estimate, which was made in an unspecified 
Giemtity, “lines of code" (LOC), was derived from a computer 
sort of S percent of the source code. 

Estimated development effort was submitted by the con- 
tractor in man-hours which are considered convertible for 
the purposes of this study to man-months (MM) at the Boehm 
rate of 152 manhours per man-month. A man-year (MY) 1s 
defined as twelve man-months. The estimates encompass all 
work charged to software from the start of the program design 
Specifications to the completion of software integration and 


test. Unfortunately the estimates for the AN/TTC-42 and the 
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SB-3865 could not be separated and the figures listed 
meappendix A are for the entire ULCS project, as noted 
in the introduction. An estimate of the effort expended 
up to the 1979 rebaselining was also obtained. Estimated 
development schedule was provided in terms of start and 
stop dates for the period covered by the estimates of 
development effort. 

Additional model inputs, unique to COCOMO or Putman, were 
required before the evaluation process could begin. In the 
case of COCOMO, information was required to compute the 
effort adjustment factor (EAF). Estimates of the cost driver 
Patanes were solicited from both contractor and NAVELEX 
software personnel, all of which corresponded closely. The 
researcher's synthesis of these estimates is contained in 
Appendix B, and yields an EAF value of 1.11. In the case of 
the Putnam model, a value for the technology constant (Cy) 
was required. After review, a value of 10040 was adopted as 
representing a conservative estimate of the advanced state 
of software engineering technology employed by ITT. This 
figure was extracted from a 1979 article (Putnam: p. 317, 
1980) and "represents an average state-of-the-art development 
environment uSing on-line interactive development." Given 
the unusual sophistication of the ITT software development 
environment the researcher feels that this figure is some- 


what conservative. 
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The evaluation of the data proceeded as follows. Since 
none of the available estimates of program size met the 
criteria for delivered source instructions (Boehm: pp. 58 §& 


59, 1981), all available estimates of program size were used 


as inputs to the COCOMO model. The estimates of development 
effort and schedule which resulted were then compared to the 


ITT estimates and a percent difference figure of the form 


given by equation (12) was generated. 


Cc 
oe 


ee = (12) 
In addition, contractor estimates for development effort and 
schedule were entered into the equations in order to obtain 
the implied values of the other software parameters. The 
results of the Basic and Intermediate COCOMO models are con- 
tained in Appendices C and D respectively. 

The Putnam model was evaluated by using ITT estimates of 
development effort and schedule with the assumed technology 
constant of 10040 as inputs to the model. The predicted 
program size was then compared to the estimated program size 
in lines of XAS-8, which was considered the metric most 
closely resembling that defined for the model by Putnam. 

The estimates were also used to generate a percent difference 
figure (A%) in the manner described above. The results of 


the Putnam evaluation are contained in Appendix E. 
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IV. MODEL APPLICABILITY AND PERFORMANCE 


The most significant factor in accounting for model per- 
formance is the amount and type of information available for 
evaluation. Thibodeau quotes another researcher on this 
subject: 

All of the data used...were data of opportunity, 
i.e., we took what we were able to get in the time 
available. Hard data on the costs of computer pro- 
gramming...are scarce commodities both in computer 
programming organizations and in the published 
literature. Few numerical data are recorded; fewer 
yet are recorded under "controlled" conditions, and 
Still fewer are suitable for generalization to other 
Slituations.... (Thibodeau: pp. 6-11 § 6-12, 1981) 

The next most important factor is the scope of the work 
to be done as measured by program size. The data which the 
models were developed to explain consist of completed pro- 
jects and, as such, the relationships to be observed among 
the data are static. In a dynamic situation like the 
AN/TTC-42 program, relationships among project estimating 
parameters Similar to those in the model data bases may not 
exist until the end of the development period. This 
inability to forecast project scope is particularly damaging 
in the case of staffing. Both models (Boehm: pp. 89-94, 
1981; Putnam: pp. 25-27, 1980) develop desired staffing 
levels along a time line that represents development schedule. 


Putnam provides examples of what happens when the required 


staffing levels are not maintained. One of these examples 





mest Dp. 27, 1900) tllustrates the situatt#oen that occurs 
when management adds personnel in step-like increments, but 
makes the additions only after perceived slips in development 
schedule. Instead of following the Rayleigh curve, staffing 
either exceeds requirements and labor is wasted, or there is 
too little labor available and the program slips. The 
researcher believes that, in effect, this situation is similar 
to the AN/TTC-42 program in which the amount of work required 
is constantly increasing. Staffing 1s rarely at the optimal 
value, because the correct value of program size on which to 
base estimates of development effort and staffing is constant- 
ly changing and never known with certainty until after the 
fact. The Putnam model provides a trade-off mechanism for 
time and development effort, but this is not an open-ended 
relationship. It is bounded below by ehte diffticwudty gradient, 
SO that, at some point the schedule stretchout caused by 
increases in project size cannot be overcome by adding more 
people. 

In addition to uncertainty concerning the size of the 
program, the units used to measure Size are also a problem. 
ITT's method of accounting for program size in bytes does 
not meet the requirements of the models. Unfortunately, 
neither do any of the other measures of program size that 
are readily available. The metric most similar to the 
delivered source instructions and source statements required 


would seem to be lines of XAS-8 assembly code. However, 
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this is deceptive. While it is true that XAS-8 is what the 
programmers write, a prime requisite of the models, it is 
not the language in which they test and debug. \XAS-8 input 
to the macro assembler produces an output of 8080A machine 
instructions which actually execute on the processors. 
Unlike the case in which a higher order language (HOLL) is 
used, debugging for the AN/TTC-42 is not done in the HOL, 
but in 8080A instructions. No mechanism is mentioned in 
the models for handling such an anomaly. 

An important aspect of program size not easily assessed 
1s the effect of the control tables used in this real-time, 
event-driven system. Putnam does not address the handling 
of data elements, and Boehm approaches them from the point 
of view that if a programmer writes a data declaration it 
counts as a delivered source instruction. (Actually, as 
moted in the discussion of his model, he considers a 
delivered source instruction to be a "card image''! which 
could contain multiple data declarations.) This manner of 
handling data does not seem applicable to the control tables. 
All concerned with the project agree that these control 
tables are extremely difficult to construct, and, as program 
Size has expanded, so has the size of the tables. From a 
1977 estimate of 22 kbytes, the tables have expanded to a 
Beoeecstimate Of 53 kbytes. This 138 percent increase in 


the most difficult part of the programming effort may have 





had a much greater effect on the project than would the same 
increase in the number of lines of XAS-8, particularly if 

the tables became increasingly complex in an effort to prevent 
the program from exceeding the main storage constraint. 

The COCOMO cost driver rating for the main storage 
constraint is "Very High" but still understates the effect 
that it has had on the program. To deliver a functional 
product the contractor clearly must provide sufficient 
memory to accommodate the program and data. The AN/TTC-42 
main memory storage capacity and estimates of combined 
program and data storage requirements for each data point 
are contained in Appendix A. (Note: These figures accord 
with the Boehm definition of main storage [Boehm: p. 410, 
1981], and do not include data stored in a bulk storage 
unit which was part of the 1977 and 1979 estimates.) While 
the main memory constraints indicated in Appendix A would not 
seem particularly significant, they must be viewed in light 
of the continued growth of the program and data base over the 
development period. Although at the time the project started 
in 1977 the programmers had an 18 percent reserve of memory, 
this hand dwindled away to 2 percent by the time of the 1979 
rebaselining. 

If the programmers had been uninhibited by the approaching 
exhaustion of memory resources during the 1977-1979 period 
this would have had little effect on the project, but the 


researcher considers this to have been extremely unlikely. 
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Instead, it is more likely that, as the memory reserve 
eroded, the contractor would have placed increasing emphasis 
on utilizing the most memory-efficient methods possible to 
implement program tasks. Presumably, this constraint 
increased the difficulty of programming system routines and 
may have led to increased complexity and size of the control 
tables as a memory conservation measure, Effort may also 
have been expended reworking already completed routines to 
compress their memory requirements. The point here is not so 
much the value of the main storage constraint at the start 

of the baseline, but rather the increasing pressure placed on 
the project personnel to fit the expanding program and data 
into available memory as the hardware redesign limit was 
approached. More than likely this process was repeated 
during the 1979-1981 time frame. 

Another factor adversely affecting model performance is 
the manner in which the Intermediate COCOMO effort adjustment 
factor was estimated. The cost driver ratings used for the 
evaluation represent average ITT values over the life of the 
project. This situation is complicated by the presence of 
more than one software development organization. The ITT 
portion of the programming effort, with which the personnel 
interviewed were most familiar, was confined to the switching 
control processor. ITT subcontracted the development of the 
MCPU software, the software tools, and the environmental 


Simulator. No estimate of the competence of the subcontractor 
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personnel was obtained, and, had one been available, an 
attempted synthesis of the subcontractor and ITT personnel 
values would have produced a figure of little significance. 
However, if this data could have been coupled with suffi- 
clent program structure information in order to use the 
Detailed COCOMO model, the results might have been quite 
different than those produced by the Intermediate model. 

In addition to lack of knowledge of the ITT subcontractor 
personnel qualifications, the researcher was not able to 
obtain estimates of the programming effort required to 
produce the software tools and the environmental simulator 
in use on the project. This effort must have been substantial 
given the sophistication of the tools described in the intvro- 
duction, and it is clearly included in Boehm's model by his 
definition of deliverable source instructions. The inclusion 
of this data in the Boehm model inputs might have substantial- 
ly reduced the variance between the ITT and COCOMO effort 
peeatctions. 

Other difficulties encountered in applying the models 
came trom determining the technology constant and the appro- 
priate time units to use with the Putnam model. No explicit 
guidance is provided by Putnam for determining the value of 
the technology constant. If, as Putnam states, the tech- 
nology constant is to be determined from the past performance 
of the organization, then it encompasses considerably more 


than "the use of modern programming practices, the language 
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used, the development environment (on-line, interactive 
development versus batch), and the availability of the develop- 
ment machine," that he mentions. In some way it is measuring 
the overall software development capability of the organiza- 
tion, and this use of the technology constant makes it what 
Thibodeau (Thibodeau: p. A-69, 1981) terms a self-calibrating 
model. This is particularly significant for model performance 
as Thibodeau found in a study of eight software cost esti- 
mating models that the factor most significant in increasing 
model accuracy was calibration to the development enviornment 
(Thibodeau: p. 1-8, 1981). In view of this, the lack of a 
technology constant actually based on the ITT development 
environment must be considered a serious shortcoming of the 
modeling process. 

The difficulty with the time scale arose from Sea eHenis 
Pyeeutnam (Putnam: pp. 5 § 53, 1980) that the input to his 
model may be measured in a variety of units. The examples 
in (Putnam, 1980) are calculated in years and man-years, and 
no mention is made of any changes to the software equation 
that are required if other units are used. For comparison 
the researcher conducted calculations with the Putnam model 
using two different time scales for the same inputs. The 
Beoults in Appendix E indicate that time units are not inter- 
changeable without the use of a different technology 


Seomacant. 
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No correlation between the results of the ITT procedure 
and the models was posited, and none appears in the results 
of the evaluation. A significant trend noted was that the 
models consistently predicted that substantially less time 
and effort should have been required to complete the project 
than was predicted by ITT for all model inputs except bytes. 
Basic COCOMO produced percent difference figures for develop- 
ment effort and schedule estimates ranging from a best case 
of minus thirteen and minus twenty-nine percent respectively 
for the 1977 Intel 8080A instruction input, to minus eighty- 
seven and minus seventy-eight percent for the 1979 PDL input. 
Intermediate COCOMO estimates closely followed those of the 
Basic model, but substantial variations were noted for the 
Putnam model. When inputs of effort in man-months and 
development time in months were used with this model, 
estimates of program size exceeding those made by the con- 
muaetOr Dy several thousand percent resulted. In addition, 
when the effort expended up to the 1979 rebaselining (190 
man-months) is input to the Basic COCOMO model, a value of 
27251 delivered source instructions results. This correlates 
quite well with the 29498 lines of XAS-8 estimated from the 
rebaselining proposal, but ITT estimated a development 
schedule of forty-eight months while the model predicted 
that the 27 KDSI should have been developed in thirteen 


months. 





V. CONCLUSIONS 


Little is likely to be accomplished using the COCOMO 
and Putnam models on software development projects like the 
AN/TTC-42 until the size of the program that has to be 
developed can be estimated with reasonable accuracy or at 
least fixed between useful limits. (But see [Thibodeau: 

p. 5-29, 1981] for the performance of a model that does not 
use program size as an input.) Even were accurate estimates 
of this and other estimating parameters available, however, 
the utility of the models would still be severely limited 
by the differences between the inputs as they are defined 
in the models and as they are available in the development 
environment. Where the parameters are clearly defined 
(COCOMO) this problem is amenable to solution by software 
development personnel. However in the case of the Putnam 
model the parameters are not sufficiently well defined, 
with the possible exception of the program size parameter, 
to provide a useful guide for a data collection effort. 

In particular, the means of determining the technology 


constant used with this model are completely inadequate. 
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APPENDIX A 


DATA 
fo 7 
Program Size 
Bytes 91,443 
S080A Instructions 50,802 
XAS -8 Tape) NS 


Lines of Code (Crandell) 
Software Development: 


Eo rt (ULCS) 


Man-hours 70.195 
Man -months 462 
Man-years 8); 


Schedule (AN/TTC-42) 


Months ty 
Years Va 


Data (Bytes) 


Total 105,494 
Control Tables Pai 0.10. 5) 
Bulk Storage 56,000 
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116,640 
64,800 
37,626 


100,682 
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22,780 
54.700 
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234,750 
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75,726 
100,000 


154,166 
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APPENDIX B 


EFFORT ADJUSTMENT FACTOR 


Rating Brtort Multerolier 

Product Attributes 

Relay Very High eeu) 

DATA LOw 94 

CPLX Very High Lav 
Computer Attributes 

TIME Nominal 1010 

STOR Very High i ra 

VIRT Low Poy) 

TURN Low 87 
Personnel Attributes 

ACAP High . 86 

Pe x P High aoa 

PCAP Nominal 8:0 

Vee Very Low Ibe 4 dk 

EX? Nominal oo 
Project Attributes 

MODP Very High Bes 

TOOL Very High Bron) 

SCED Very High Po) 


Effort Adjustment Factor = 1.11 


46 


APPENDIX C 
BASIC COCOMO 


In this appendix, the software development parameters 
have been generated using the data in the left hand column 
as inputs to the Basic COCOMO estimating equations, equations 
Meawa (4). The inputs are either ITT estimates (Bytes, 
PDL, development effort and development schedule) or are 
generated from ITT estimates (8080A instructions and XAS-8) 
as described in the text. To simplify comparison of the 
estimates of software development effort and schedule pro- 
duced by the BASIC COCOMO model and those estimated by ITT, 
a percent difference figure is calculated for all cases 
except 1981 development effort for which no ITT estimate was 
available. In addition, where ITT estimates of software 
development effort or schedule are used as inputs, an implied 


value for program size is generated. 
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APPENDIX D 
INTERMEDIATE COCOMO 


The input parameters used with the Basic COCOMO model 
are here employed with the Intermediate COCOMO estimating 
equations, equations (7) and (8). As no adapted software is 
used in the AN/TTC-42 programs, the values for KEDSI and 
KDSI are equal. The Effort Adjustment Factor is calculated 
in Appendix B, and was found to be 1.11. Percent difference 
figures for the Intermediate COCOMO and ITT estimates are 
calculated in the same manner as in Appendix C, and again, 
no comparison of estimated software development effort at 
the 1981 rebaseline point was possible for lack of an ITT 


value, 
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APPENDIX E 
THE PUTNAM MODEL 


In this appendix the Software Equation, equation (9), is 
used to generate estimates of program size in source state- 
ments (S.) from an assumed technology constant (C,) and ITT 
estimates of development effort (E) and schedule (to) in 
two time scales. The estimated number of source statements 
1s compared to the number of lines of XAS-8 source code 
estimated to have been required and a percent difference 
figure is calculated for these values. Lack of a 1981 ITT 
effort estimate prevented a program size estimate from being 


made at that point. 
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